
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 223 1 3- 1 450 
www.usplo.gov 



| APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOOCET NO. 


CONFIRMATION NO. 


09/581,004 


07/17/2000 


SHUSAKU OKAMOTO 


MTS-3200US 


2255 



7590 07/24/2006 

ratner&prestia 

one westlakes berwyn suite 301 

PO BOX 980 

VALLEY FORGE, PA 19482-0980 



EXAMINER 



VO, TUNG T 



ART UNIT 



PAPER NUMBER 



2621 

DATE MAILED: 07/24/2006 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 


Application No. 

09/581,004 


Applicant(s) 

OKAMOTO ET AL. 


Examiner 

Tung Vo 


Art Unit 

2621 





« The MAILING DATE of this communication appears on the cover sheet with the correspondence address 



Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 

WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )E3 Responsive to communication(s) filed on 27 June 2006 . 
2a)Q This action is FINAL. 2b)^ This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1, 453 O.G. 213. 

Disposition of Claims 

4) IEI Claim(s) 1.5,8.9,11 ,12,1 4-20.24.26.29.30.32.37 A3 and 44 is/are pending in the application. 

4a) Of the above claim(s) 2-4.6.7.1 0,1 3.21 -23.25,27.28.31. 33-36 and 38-42 is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1.5.8.9. 1 1. 12.14-20.24.26.29,30.32 and 37 is/are rejected. 

7) £3 Claim(s) 43 and 44 is/are objected to. 

8) \Z\ Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)|EI All b)D Some * c)D None of: 

1 Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) □ Notice of References Cited (PTO-892) 

2) O Notice of DraftspersorVs Patent Drawing Review (PTO-948) 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 

Paper No(s)/Mail Date . 



4) □ Interview Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) O Notice of Informal Patent Application (PTO-152) 

6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 7-05) 



Office Action Summary 



Part of Paper No./Mail Date 20060713 



Application/Control Number: 09/5 8 1 ,004 Page 2 

Art Unit: 2621 

DETAILED ACTION 
Continued Examination Under 37 CFR LI 14 
1 . A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.114. Applicant's submission filed on 06/27/2006 has been entered. 

Claim Rejections - 35 USC § 103 
1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

1. Claims 1, 5, 8-9, 1 1-12, 14-20, 24, 26, 29-30, 32, and 37 rejected under 35 U.S.C. 103(a) 
as being unpatentable over Ejiri et al. (US 5,969,969) in view of in view of Kumamoto Kenji (JP 
10-257482). 

Re claims 1 and 37, Ejiri teaches a vehicle-operation assist for a vehicle (100 of figs. 33 
and 37) mounted with a camera (120, 124 of fig. 33), said vehicle operation assist comprising: an 
imaging unit (120, 124, 125 of fig. 33) operable to generate a circumferential state image using 
an output image from the camera (124 of fig. 33), said circumferential state image viewed from 

synthetic-image generating unit (1 17 of fig. 33) operable to generate a synthetic image 
using the circumferential- state image (126a, 1 15a of fig. 33) and an assumed-movement pattern 
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image (1 14b and 1 16c of fig. 33) which is based on a predetermined driving operation of the 
vehicle (1 18 of fig. 33) and a display unit (160 of fig. 33) operable to display the synthetic image 
(figs. 37-42), the assumed-movement pattern image including a movement of the vehicle (the 
arrow shows forward movement of the vehicle (A, B, C of fig. 39) between a start position of the 
vehicle (when the vehicle prepares for parking, there would be a start position, 100 start position, 
the vehicle (100 of fig. 39) is not in a park position yet) and a parking position of the vehicle (the 
vehicle completely parked) in case of performing the predetermined driving operation (706, 707, 
708 of fig. 38). 

It is noted that Erjiri does not particularly teach a virtual viewpoint, said virtual viewpoint 
being different from a viewpoint of the camera as claimed. 

However, Kenji teaches the synthetic image (42 of fig. 5-8) showing the car (46 of fig. 5) 
from a virtual point of view as three dimensional space (8 of fig. 8) that is converted from the 
actual viewpoint (6, 10, 1 1, 14, and 15 of fig. 2; 37 of fig. 3) so this suggests a virtual viewpoint 
as a three-dimensional space, said virtual viewpoint being different from a viewpoint of the 
camera. 

Therefore, taking the teachings of Erjiri and Kenji as a whole, it would have been 
obvious to one skill of ordinary skill in the art to incorporate the teachings of Kenji into the 
image processing apparatus of Erjiri for displaying a generated synthetic image showing the 
vehicle from a virtual point of view using the real image captured from cameras. 

Doing so would provide an always clean car circumference image can be seen by 
detecting the variation rate of the car body by the irregularity of a road surface by the 
acceleration sensor, feeding back to a display image, and removing the shake of an image. 



Application/Control Number: 09/58 1 ,004 Page 4 

Art Unit: 2621 

Re claims 5, 8, 9, Erjiri further teaches the circumferential state image viewed from a 
predetermined viewpoint as a point fixed three-dimensional space or the vehicle (fig. 39) 5 and the 
viewpoint converting unit is included that changes the predetermined viewpoint automatically or 
through an input from a driver (118 of figs. 33 and 37); wherein the assumed-movement pattern 
image includes virtual poles arranged on the outer edge of the vehicle movement area (A, B, C, 
and D of fig. 39); the synthetic-image generating unit superimposes a current-position camera in 
which the vehicle is present, on the circumferential-state image to generate the synthetic image 
(D and A of fig. 39). 

Re claims 1 1 and 12, Erjiri further teaches wherein when actual driving operations 
corresponding to predetermined series of driving operations are started (130 of figs. 33 and 37; 
see also fig. 38), the synthetic-image generating unit fixes positional relation between the 
assumed-movement pattern image and the circumferential-state image at the point of time when 
the actual driving operations are started and generates the synthetic image (706, 707 and 708 of 
fig. 38); a positional-information storing unit is included which stores positional: information of 
the whole or a part of the assumed-movement pattern image with regard to the whole or a part of 
the video data for the circumferential-state image on the synthetic image when the actual driving 
operations are started (116 of figs. 33 and 37), and; the synthetic-image generating unit (117 of 
figs. 33 and 37) fixes the positional relation in accordance with the positional information. 

Re claims 14-20, Erjiri further teaches wherein a final-position inputting unit for 
inputting a final position which is a position of the vehicle at end of the movement and a start- 
position determining unit for obtaining a start position which is a position at start of the 
movement corresponding to the input final position in accordance with the-an assumed- 



Application/Control Number: 09/581,004 Page 5 

Art Unit: 2621 

movement pattern are included (709 of fig. 38), and the synthetic-image generating unit 
superimposes the input final position and the start position corresponding to the input final 
position on the circumferential-state image to generate the synthetic image (fig. 39); a start- 
position guiding unit (1 18a of fig. 33 and 37) is included which guides the vehicle to the start 
position by automatically controlling driving of the vehicle; an assumed-movement-pattern 
storing unit (1 16b of figs. 33 and 37) holds a plurality of assumed-movement patterns; the 
assumed-movement-pattern storing unit holds a plurality of assumed- movement patterns (1 16b 
of fig. 33 and 37), and a pattern selecting unit (118 of figs. 33 and 37) is included which 
automatically selects the assumed- movement pattern through an input from a driver or a 
predetermined driving operation; wherein the pattern correcting unit (709, 710, and 711 of fig. 
38) is included which updates and corrects content of the assumed-movement pattern stored in 
the assumed- movement-pattern storing unit; wherein the pattern correcting unit updates and 
corrects the assumed- movement pattern (140 of fig. 37, Note updates and corrects the current 
positions of the vehicle) in accordance with the vehicle positions at start and end of the corrected 
movement input from a driver (NOTE the guidance information selector (1 18of figs. 33 and 
37)then selects the most proper guidance information (1 18a of figs. 33 and 37) for the driver 
from the viewpoint of safety, allowance, etc. according to whether or not it is possible for the 
driver to judge the information subjectively. The selected result is output onto the display unit 
(160 of figs. 33 and 37), such as an instrument panel, console panel and so on); wherein the 
pattern correcting unit updates and corrects the assumed- movement pattern in accordance with 
an actual driving operation (. 
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Re claims 24, 26, 29, 30, and 32, Erfiri further teaches wherein when the viewpoint 
converting unit changes the predetermined viewpoint, the viewpoint converting: unit fixes the 
predetermined viewpoint to the vehicle before the actual driving operations corresponding to the 
predetermined series of driving operations are started (fig. 40) and wherein the assumed- 
movement pattern image includes a circumscribed area on a space through which the vehicle 
passes when the predetermined series of driving operations are performed (fig. 39); wherein the 
assumed-movement includes a change from a backward movement to a forward movement or 
from a forward movement to a backward movement in the predetermined series of driving 
operations (fig. 40); wherein an obstacle inputting unit is included which is able to input a 
position of an obstacle area in an image to the display unit for displaying the synthetic image( 
fig. 34, elements x and 100b); wherein the pattern correcting unit updates and corrects the 
assumed-movement patterns in accordance with a position of an obstacle area input from a driver 
(JUDGMENT of fig. 35). 

Response to Arguments 
2. Applicant's arguments filed 06/27/2006 have been fully considered but they are not 
persuasive. 

The applicant argued that Ejiri does not disclose a synthetic image that provide two 
difference images superimposed together on as single image, and a start position of the vehicle 
and a parking position of the vehicle, page 9 of the remarks. 

The examiner respectfully disagrees with that applicant. It is submitted that the claimed 
feature calls for generating a synthetic image using the image of the vehicle and the assumed 
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movement pattern of the vehicle as taught by Ejiri (fig. 39, Note the image of the vehicle 100 and 
the movement pattern (B of fig. 39) (see also figs 34-35), and Ejiri suggests the assumed- 
movement pattern image including a movement of the vehicle (the arrow shows forward 
movement of the vehicle (A, B, C of fig. 39) between a start position of the vehicle (when the 
vehicle prepares for parking, there would be a start position, 100 start position, the vehicle (100 
of fig. 39) is not in a park position yet) and a parking position of the vehicle (the vehicle 
completely parked) in case of performing the predetermined driving operation (706, 707, 708 of 
fig. 38), wherein the display (160 of fig. 33, see also fig. 39) for displaying the synthetic image 
(figs. 39-42). 

It is acknowledged that the pointed out figure 7 shows the synthetic image includes the 
surrounding space around the vehicle plus the assumed movement of the vehicle, these features 
are not claimed. Therefore, the argument is not persuasive. 

Allowable Subject Matter 

3. Claims 43 and 44 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the base 
claim and any intervening claims. 

Conclusion 

4. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. See the previous Office Action 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Tung Vo whose telephone number is 571-272-7340. The 
examiner can normally be reached on Monday-Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on 571-272-7418. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272JJ1QQ 




Tung Vo 
Primary Examiner 
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